A prehospital risk assessment tool predicts clinical outcomes in hospitalized patients with heat-related illness: a Japanese nationwide prospective observational study

We previously developed a risk assessment tool to predict outcomes after heat-related illness (J-ERATO score), which consists of six binary prehospital vital signs. We aimed to evaluate the ability of the score to predict clinical outcomes for hospitalized patients with heat-related illnesses. In a nationwide, prospective, observational study, adult patients hospitalized for heat-related illnesses were registered. A binary logistic regression model and receiver operating characteristic (ROC) curve analysis were used to assess the relationship between the J-ERATO and survival at hospital discharge as a primary outcome. Among eligible patients, 1244 (93.0%) survived to hospital discharge. Multivariable logistic regression analysis revealed that the J-ERATO was an independent predictor for survival to discharge (adjusted odds ratio [OR] 0.47; 95% confidence interval [CI] 0.37–0.59) and occurrence of disseminated intravascular coagulation (DIC) on day 1 (adjusted OR 2.07; 95% CI 1.73–2.49). ROC analyses revealed an optimal J-ERATO cut-off of 5 for prediction of mortality at discharge (area under the curve [AUC] 0.742; 95% CI 0.691–0.787) and DIC development on day 1 (AUC 0.723; 95% CI 0.684–0.758). The J-ERATO obtained before transportation could be helpful in predicting the severity and mortality of hospitalized patients with heat-related illnesses.


Scientific Reports
| (2023) 13:1189 | https://doi.org/10.1038/s41598-023-28498-z www.nature.com/scientificreports/ rate ≥ 100 bpm, body temperature ≥ 38 °C, and age ≥ 65 years), with a total score ranging from 0 to 6 (Supplemental Table 1). We demonstrated that the J-ERATO score is accurate in predicting ICU admission and in-hospital mortality 11,12 . Furthermore, the usefulness of the J-ERATO score has been successfully validated in South India, which has a population with a different age distribution to Japan 13 . Early recognition of and treatment for heatstroke can reduce morbidity and mortality irrespective of its etiology 8,14,15 . Rapid whole-body cooling followed by advanced care in the ICU is essential to improve survival after exertional heatstroke [16][17][18] . As most cases of heat-related illness are encountered in the prehospital phase in emergency care settings, rapid recognition, and assessment of patients with clinical manifestations of heatstroke in the prehospital setting play pivotal roles in the timely initiation of interventions. The best practices require rapid cooling of the patient to a temperature less than the threshold for critical cell damage (~104.5°F) in less than 30 min from the time of collapse. Cold water immersion is considered the gold standard treatment, but unfortunately, there are many situations in which this method is unavailable 17 . Clinical prediction scores and laboratory measures related to outcomes could be useful to prepare for possible clinical deterioration. However, no precise clinical score or biomarker has been adopted for the prediction of clinical deterioration and prognosis in patients hospitalized for heat-related illnesses. To address this clinical gap, we evaluated the relationship between the J-ERATO score and clinical deterioration after hospital admission in a cohort of patients who required hospitalization for heat-related illnesses.
The purpose of this study was to evaluate the ability of the J-ERATO score, which was obtained by emergency medical service (EMS) personnel at the scene, to predict mortality and the development of DIC and multiorgan dysfunction among patients hospitalized for heat-related illnesses.

Methods
Study design and settings. This analysis was conducted using a registered database of a prospective, multicenter, observational study (the Heatstroke Study) in Japan from July 1st to September 30th in 2019, 2020, and 2021. Briefly, the Japanese Association for Acute Medicine (JAAM; Heatstroke and Hypothermia Surveillance Committee) established the Heatstroke Study in 2006. It involved a survey of patients with presumed heatrelated illnesses who were transferred to emergency hospitals by EMS personnel 11,[19][20][21][22][23] . Since 2017, this registry has been used to gather information regarding only patients admitted to the hospital owing to a diagnosis of heat-related illness. Approximately 142 emergency hospitals from all over Japan took part in the registry during the study period 12 . Data were manually recorded by the staff at each participating hospital by using standardized record sheets. The study protocol was approved by the Teikyo University Ethical Review Board for Medical and Health Research (approval number; 17-021-5, board name; "Heatstroke STUDY", approval date; May 21st, 2020), which waived the requirement for informed patient consent owing to participant anonymity. The procedures followed were in accordance with the ethical standards of the institution's responsible committee on human experimentation and with the Helsinki Declaration of 1975, as most recently amended.

Definition of heat-related illnesses.
Heat-related illnesses were defined as conditions in the spectrum of illnesses progressing from heat exhaustion to heatstroke during exercise or exposure to environmental heat stress. Heat-related illness was diagnosed by the attending physician in the emergency department where the patient was admitted.
Selection of participants. This study included adult patients (aged ≥ 18 years) with a diagnosis of heatrelated illness who were admitted to the hospital. The heat-related illness was diagnosed by the modified definition of heatstroke (mJAAM) criteria 19 . Briefly, the mJAAM criteria consist of factors such as CNS manifestations, hepatic/renal dysfunction, and DIC, but not body temperature. Patients who were not directly transferred from the occurrent site to the participant's hospital, for whom transportation data were missing, or who suffered cardiopulmonary arrest on arrival were excluded from this study. In addition, patients for whom data on the primary outcome, J-ERATO score, event location, and/or the circumstances of the occurrence were missing were excluded from the analyses. Data collection. Patient demographics, prehospital information collected by the EMS personnel, and inhospital information were prospectively recorded (i.e., event location, circumstances of the occurrence, transportation, pre-existing functional dependency, age, sex, prehospital vital signs [Glasgow Coma Scale score, pulse rate, non-invasive blood pressure, respiratory rate, body temperature, and peripheral pulse oximetry], medical history [liver disease, cerebrovascular disease, respiratory disease, chronic kidney disease, immunocompromised disease, psychological disorder, diabetes mellitus with organ dysfunction, and previous heat-related illness], physical findings assessed by the EMS at the scene (seizure, dry skin, skin redness, and skin hotness to touch), in-hospital information [the location of admission, i.e., the ICU or a non-ICU general ward), and survival to hospital discharge. In addition, we gathered information about the patients' Sequential Organ Failure Assessment (SOFA) score 24 on the first day after admission (day 1) and the presence of DIC on day 1. Owing to the lack of a specific mortality prediction tool for heat-related illnesses, the SOFA score, which is a general scoring system for critically ill patients, is commonly used to estimate the severity of heat-related illnesses 25 . DIC was diagnosed according to the JAAM DIC diagnostic criteria 26,27 , with a total score ≥ 4 establishing a diagnosis of DIC. Scoring is based on peripheral blood platelet counts, prothrombin time, fibrinogen/fibrin degradation products or D-dimer levels, and the presence of systemic inflammatory response syndrome.
Outcomes. The primary outcome was survival to hospital discharge. The secondary outcomes were occurrences of DIC and SOFA score on day 1. www.nature.com/scientificreports/ Statistical analysis. Baseline characteristics were summarized using medians and interquartile ranges for continuous variables and frequencies (%) for categorical variables. The differences between groups were tested using the Mann-Whitney U test. The chi-square or Fisher's exact test was used to compare binary variables. We evaluated the relationship between the J-ERATO and SOFA scores using Spearman's rank correlation test. Multivariable logistic regression was used to adjust for the selected covariates to determine whether the J-ERATO is related to the prognosis of patients hospitalized for heat-related illnesses. The following independent variables were selected for the model: the J-ERATO score, event location, circumstances of the occurrence, medical history of psychiatric disorder, and age. A set of selected covariates was chosen a priori based on biological plausibility and a priori knowledge [28][29][30][31][32] .
We calculated the areas under the receiver operating characteristic curve (AUCs) and their 95% confidence intervals (CIs) to evaluate the predictive ability of J-ERATO score to differentiate between survival and nonsurvival at hospital discharge, and between patients with and those without DIC on day 1. The best cut-off point for the J-ERATO score to determine the highest mortality risk was based on Youden's index. We also performed multivariable analyses in the subgroup of ICU-admitted patients, adjusting for the same variables as in the overall analyses.
All statistical analyses were carried out with a two-sided significance level of 5% via JMP PRO 16.0.0 (http:// www. jmp. com).

Results
Characteristics of study subjects. A total of 2474 hospitalized patients diagnosed with heat-related illnesses were registered during the study period. After excluding 1137 patients, 1337 patients were analyzed in this study (Fig. 1).
Of the 1337 eligible patients, 1244 (93.0%) survived to hospital discharge. Table 1 summarizes the patient characteristics of the survivor and non-survivor cohorts. Survivors were younger (P = 0.03) and more commonly experienced events outdoors (P < 0.0001) than non-survivors. Compared with survivors, non-survivors exhibited a significant deterioration of prehospital vital signs, with lower systolic and diastolic blood pressures and SpO 2 , and a higher pulse rate, respiratory rate, body temperature, and disorientation rate (all P < 0.0001). Non-survivors had a higher proportion of seizures at the scene than did survivors. Table 2 summarizes the duration of hospitalization, outcomes, and severity scores among the survivors and non-survivors. Survivors had a lower proportion of ICU admission (P < 0.0001) than non-survivors. Among the patients admitted to ICU (560 of 1337 eligible patients [41.9%]), survivors had significantly longer ventilator-free (P < 0.0001) and ICU-free (P < 0.0001) days than non-survivors. On day 1, fewer survivors had DIC (P < 0.0001), and survivors had lower SOFA scores (P < 0.0001) than non-survivors. Table 3 demonstrates the association between the J-ERATO score and primary outcome among all patients. The J-ERATO score was an independent negative predictor of survival at hospital discharge.   Table 4 demonstrates the association between the J-ERATO score and the occurrence of DIC among all patients. The J-ERATO score was an independent positive predictor of DIC on day 1.

Main results.
We assessed the heterogeneity of our results by conducting subgroup analyses of ICU-admitted patients. The subgroup results were similar to the main results, emphasizing the role of the J-ERATO score on both survival outcome at discharge and development of DIC on day 1 (Supplemental Table 2).
We further evaluated the sensitivity and specificity of J-ERATO score cut-off values for the entire cohort. The receiver operating characteristic analyses revealed a J-ERATO score cut-off of 5 points, providing optimal sensitivity and specificity to predict mortality at discharge (AUC, 0.742; 95% CI 0.691-0.787; sensitivity, 74.2%; specificity, 64.4%) (Fig. 2, left). For calibration, the predicted probability and actual observation by score are illustrated (Fig. 2, right).
In this study, a large amount SOFA score data was missing, both in the entire cohort (29.9% [400 of 1337]) and the ICU-admitted cohort (missing, 25.0% [140 of 560]). Figure 4 shows the association between the J-ERATO and SOFA scores on day 1 in the entire cohort. An increased J-ERATO score was significantly associated with an increase in the SOFA score in both cohorts (both P < 0.0001). There was a positive association between the J-ERATO and SOFA scores on day 1 (Spearman's rank correlation coefficient, 0.418; P < 0.0001). Subgroup analysis of ICU-admitted patients revealed a similar association between the J-ERATO and SOFA scores (Supplemental Figure 1).
We addressed the issue of missing SOFA score data by conducting additional analyses. Accordingly, there were no differences in age, event location, occurrence circumstances, prehospital blood pressure, ICU admission rate, or mortality between patients with and those without SOFA data. Patients without SOFA data had a lower body temperature, a lower proportion of prehospital CNS abnormalities, and a lower incidence of ICU admissions (Supplemental Table 3).

Discussion
This study verified the clinical usefulness of the J-ERATO score 11 for patients with heat-related illnesses who were transferred to the hospital by EMS personnel. To the best of our knowledge, this is the first study to demonstrate that the J-ERATO score is an independent predictive marker for survival to discharge as well as the occurrence of DIC on day 1, and that there is a significantly positive correlation between the J-ERATO score and the severity of organ dysfunction (the SOFA score) on day 1. Given that the J-ERATO score is calculated with only physical information regarding prehospital vital signs and patient age, our observations are very important to improve outcomes in patients with heat-related illnesses.  www.nature.com/scientificreports/ While risk factors for mortality have been investigated in patients admitted to the ICU with heatstroke [33][34][35] , few studies have focused on early screening tools for the prediction of clinical outcomes in patients with heatrelated illnesses in the prehospital setting, emphasizing the importance of the J-ERATO score. Furthermore, the J-ERATO score has been validated in South India 13 . Ninan et al. reported that a higher J-ERATO score at hospital presentation was an independent predictor of underlying multiple organ dysfunction syndromes, defined as a SOFA score > 2 points (P < 0.029) 13 . On the one hand, we previously reported the potential of machine learning-based mortality prediction models for heat-related illnesses (AUC, 0.92), using 24 variables at hospital arrival 22 . However, at present, such a model is not clinically feasible in most cases where heat-related illnesses are encountered, as machine learning algorithms require certain computer equipment for calculation. In general, a cumbersome tool is unlikely to be used consistently by busy physicians. Thus, a simple bedside tool for use at the time of presentation of heat-related illness is necessary. Furthermore, there is strong evidence that immediate and aggressive cooling after heatstroke ensures survival with limited sequelae 8,15,16,[36][37][38] , highlighting the need for appropriate prehospital care. Aggressive treatment in the prehospital setting according to the severity of heat-related illness, as determined with the J-ERATO score, is a promising strategy.
Although no universally accepted definition of heatstroke exists in clinical settings, Bouchama's definition is most commonly used globally 9 . They defined heatstroke as severe illness characterized by a core temperature of > 40 °C and CNS abnormalities resulting from exposure to environmental heat (classic heatstroke) or strenuous physical exercise (exertional heatstroke). However, mortality in heat-related illness is attributable not only to CNS abnormalities but also to organ dysfunction. Of note, the maximum body temperature recorded upon hospital admission after EMS transferal in several fatal cases was below 40 °C 19 . Mortality after heat-related illness increased significantly to approximately 5% at body temperatures above 38.1 °C 19 . To address this clinical weak point in Bouchama's definition, the JAAM first established criteria for heat-related illnesses in 2014 and modified the definition of heatstroke (mJAAM) in 2016 19 . Apart from body temperature, the mJAAM criteria for heatstroke include all the components of Bouchama's criteria 21 . Kondo et al. used the previous JAAM heatstroke database in 2014 and evaluated the differences between Bouchama's criteria and mJAAM criteria in terms of diagnosis and identification of mortality. A total of 317 patients were included and divided into the Bouchama, mJAAM, and non-heatstroke groups, each consisting of 97, 302, and 15 patients, respectively. The sensitivity for death was 1.0 (95% CI 0.87-1.0) with the mJAAM criteria and only 0.29 (95% CI 0.14-0.49) with Bouchama's criteria. On the other hand, both the Bouchama and mJAAM criteria could not predict in-hospital mortality (AUC: 0.52 for both criteria). The median SOFAs were 5 and 3 in the Bouchama and mJAAM criteria, respectively 21 .
Taken together with our current results, the diagnosis of heatstroke with the mJAAM criteria in combination with prognostication with the J-ERATO score, especially more than 5 points of J-ERATO score, could enable physicians to optimize treatment and allocate proper medical resources to patients with heat-related illnesses.
DIC has a reported incidence of ≥ 48% among patients with heatstroke 10,39,40 . In the current study, DIC was diagnosed based on the JAAM DIC diagnostic criteria, with a total score ≥ 4 indicating a diagnosis of DIC 26,41 . Gando et al. reported that the JAAM DIC scoring system exhibited superior prognostic value over the International Society on Thrombosis and Haemostasis' overt DIC scoring system for the prediction of multiple organ dysfunction syndromes and poor prognoses in patients with severe sepsis 41 . Hifumi et al. reported that mortality worsened significantly as the JAAM DIC score increased and was approximately 10% even at a DIC score of only 2 in patients with heatstroke according to the mJAAM criteria 19 . In the current study, higher J-ERATO scores were associated with a higher prevalence of DIC and mortality. The AUC of the J-ERATO cut-off of 5 was greater than 0.7, which is regarded as an acceptable predictive accuracy 42 . Taken together, early evaluation of the patient's status should be helpful in evaluating the severity of heatstroke and determining the optimal treatment at the patient's presentation.
The current study had several limitations. First, 594/2474 (24.0%) patients in the registry were excluded from the analyses because of missing data, which may confer a risk of bias and affect our results. Especially, the high percentage of missing data may have affected the results of the AUC value. Further evaluation will be needed. Further, 512/1337 (38.3%) patients had missing data regarding the occurrence of DIC and 400/1337 (29.9%) regarding the SOFA score on day 1, for unknown reasons. We addressed the latter issue by comparing parameters between patients with and those without SOFA score data. Our results indicate that the usefulness of the J-ERATO score may be limited to patients with more severe heat-related illness. Second, we did not impute missing values because of the large amount of missing data that would need to be imputed, which might have generated bias. Thus, our findings should be interpreted with caution. Third, although neurological sequelae after heatstroke are clinically important complications 43 , we were not able to analyze the association between the J-ERATO score and neurological dysfunction because of the small number of events. Thus, the role of the J-ERATO score on neurocognitive function after heat-related illness remains undetermined. Fourth, although data were collected prospectively in this study, the J-ERATO score was analyzed in a retrospective manner. Therefore, we could not evaluate the usefulness of the treatment strategy according to the J-ERATO score. In particular, patients were treated according to the discretion of the physician in charge, which may have been based on clinical information, including the J-ERATO score, resulting in bias.
We found that the J-ERATO score has the potential to predict patients' severity expressing after transportation to the hospitals. Therefore, it is expected that EMS personnel calculate the J-ERATO score and start first aid, including aggressive body temperature control. On the other hand, the reconstitution of the database system to decrease the missing data may be needed in the future. Based on the new database, re-evaluating the accuracy of J-ERATO score for the outcomes despite patients' severity would be desirable. Further, the construction of new transportation systems or prehospital treatment strategies according to the J-ERATO score needs to be considered.

Conclusion
The J-ERATO score which can be determined by EMS personnel at the scene was an independent predictor of survival at hospital discharge and DIC incidence on day 1 and was positively associated with more severe organ dysfunction on day 1. The J-ERATO score may be helpful in predicting the severity and mortality of patients with heat-related illnesses.

Data availability
The datasets used and/or analyzed during the current study available from the corresponding author on reasonable request.